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Abstract. 

t3 

Biological and social networks have recently attracted enormous attention between physicists. Among several, 
two main aspects may be stressed: A non trivial topology of the graph describing the mutual interactions between 
agents exists and/or, typically, such interactions are essentially (weighted) imitative. Despite such aspects are 
widely accepted and empirically confirmed, the schemes currently exploited in order to generate the expected 
topology are based on a-priori assumptions and in most cases still implement constant intensities for links, 
i-^ Here we propose a simple shift [— 1, +1] — ► [0, +1] in the definition of patterns in an Hopfield model to convert 

frustration into dilution: By varying the bias of the pattern distribution, the network topology -which is gen- 
erated by the reciprocal affinities among agents (the Hebbian kernel)- crosses various well known regimes (fully 
connected, linearly diverging connectivity, extreme dilution scenario, no network), coupled with small world 
properties, which, in this context, are emergent and no longer imposed a-priori. 

The model is investigated at first focusing on these topological properties of the emergent network, then its 
thermodynamics is analytically solved (at a replica symmetric level) by extending the double stochastic stability 
technique, and presented together with its fluctuation theory for a picture of criticality: both a statistical me- 
chanics and a topological phase diagrams are obtained. 

Overall the picture depicted from statistical mechanics is quite intuitive: at least at equilibrium, dilution 
(of whatever kind) simply decreases the strength of the coupling felt by the spins, but leaves the paramag- 
(****) netic/ferromagnetic flavors unchanged. 

The main difference with respect to previous investigations and a naive picture is that within our approach 
replicas do not appear: instead of (multi)-overlaps as order parameters, we introduce a class of magnetizations 
on all the possible sub-graphs belonging to the main one investigated: As a consequence, for these objects a 
closure for a self-consistent relation is achieved. 
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1 Introduction to social and biological networks 



The paper is organized as follows: 

In this section we briefly introduce the reader to the state of the art in the applications of this model 
to investigation of collective effects in social and biological networks, then, in section 2, we present the 
model itself with all the related definitions. Section 3 deals with the topological analysis: Techniques 
from graph theory are the tools. Section 4 deals with the thermodynamical analysis: techniques from 
statistical mechanics are the tools. In section 5 we present our discussion and outlooks. 

Starting with a digression on social sciences, since the early investigations by Milgram [53], several efforts 
have been made to understand the structure of interactions occurring within a social system. Granovet- 
ter defined this field of science as "a tool for linking micro and macro levels of sociological theories" 
[52j and gave fundamental prescriptions; in particular, he noticed that the stronger the link between 
two agents and the larger (on average) the overlap among the number of common nearest neighbors, 
i.e. high degree of cliqueness. Furthermore he noticed that weak ties play a fundamental role acting 
as bridges among sub-clusters of highly connected interacting agents [32] [53J [53] . As properly pointed 
out by Watts and Strogatz [7T], from a topological viewpoint, the simplest Erdos-Renyi graphs is 
unable to describe social systems, due to the uncorrelatedness among its links, which constraints the 
resulting degree of cliqueness to be relatively small |I4j . Through a mathematical technique (rewiring), 
they obtained a first attempt in defining the so called "small world" graph [75]: when trying to im- 
plement statistical mechanics on such a topology their network has been essentially seen as a chain of 
nearest neighbors overlapped on a sparse Erdos-Renyi graph [651 ,24]. As the former can be solved via 
the transfer matrix, the latter via e.g. the replica trick, the model was already understood even from 
a statistical mechanics perspective (without introducing here a discussion on possible replica symmetry 
breaking in complex diluted systems 3!) [2D] ) . 

Coupled to topological investigations, even the analysis of the kind of interactions (still within a "statis- 
tical mechanics flavor") started in the past decades in econometrics and, after McFadden described the 
discrete choice as a one-body theory with external fields [50], Brock and Durlauf went over and gave a 
clear positive interaction strength to social ties [32, 40J . 

Even thought clearly, as discussed for instance in |21j , the role of anti-imitative actions is fundamental 
for collective decision capabilities, the largest part of interactions is imitative and this prescription will 
be followed trough the paper. 

Somewhat close to social breakthrough, after the revolution of Watson and Crick, biological studies 
in the past fifty years gave raise to completely new field of science as genomics [42,, proteinomics [46] 
and metabolic network investigations [59] which ultimately are strongly based on graph theorie^] [T5] . 
Furthermore graph structure appears at various levels, i.e. in matching epitopal complementary among 
antibodies giving raise to the so called " Jerne network" 58J [55J for the immune system [TS] [T] , or at 
even larger scales of the biological world: from the so far exploited micro and meso, to such a macro as 
virus spreading worldwide [25], food web [64], and much more [34] . 

In these contexts, surely there is a disordered underlying structure, but thinking at it as "completely 
random" is probably a too strong simplifying assumption. One of the strongest starting point when 

3 It is in fact well established that complex organisms share roughly the same amount of genes with simpler ones. As a 
result the failure of a purely reductionism approach (more genes — > more complexity) seems raising and interest in their 
connections, their network of exchanges, is enormously increasing. 
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dealing with random coupling is their independence: for example Blake pointed out [55] that exons 
in haemoglobin correspond both to structural and functional units of protein, implicitly suggesting a 
not null level of correlation among the "randomness" we have to deal with when trying a statistical 
mechanics approach. Not too different is the viewpoint of Coolen and coworkers (3SJ [7U] . 

From a completely different background, last step in this introduction is presenting the Hopfield model 
[56j . which, instead, is the paradigmatic model for neural networks. Even though apparently far from 
topology investigations, in the Hopfield model there is a scalar product among the bit strings (the 
Hebbian kernel [55J ) : despite fully connected, the latter can be seen as a measure of the strength of the 
ties (which in that context must be both positive and negative as, in order to share statically memories 
over all neurons [TU], it must use properties of spin glasses [T7][22]E2] as ^ ne k- ev f° r having several 
minima in the fitness landscape). By varying tunable parameters (level of noise and amount of storage 
memories) the Hopfield model displays a region where is paramagnetic, a region where is a spin glass 
and a region where is a " working memory" |12) . 

We are ready to introduce our starting idea: what happens if instead of using positive and negative 
values for the coupling in the Hebbian kernel of the Hopfield model, we use positive and null values? 
We want to show that, even in this context, by varying the tunable parameters, we recover several 
topologies (on which ferromagnetic or paramagnetic behaviors may arise): fully connected scenario 
weighted and un-weighted, Erdos-Renyi graphs, linearly diverging connectivity, extreme dilutions, small 
world features, fully disconnected (that is no edges at all). 

Despite a rich plethora of phenomena in graph theory is obtained, from equilibrium statistical mechanics 
perspective we find that all these networks behave not drastically differently, relating strong differences 
in dynamical features (in agreement with intuition), on which we plan to investigate soon. 



2 The model: Definitions 



Let us consider V agents ±1 3 G (1, V). In social framework (e.g. discrete choice in econometrics) 
for example di = +1 means that the i th agent agrees a particular choice (and obviously disagreement 
in the —1 case). In biological networks, i may label a Kauffman gene (assuming undirected links) or a 
Jerne lymphocytes in such a way that ai = +1 represents expression or firing state respectively, while 
quiescence is assumed when oi = —I. 

The influence of external stimuli, representing e.g. medias in social networks or environmental variations 
imposing phenotypic changes via gene expression in proteinomics or viruses in immune networks, can 
be encoded by means of a one-body Hamiltonian term H = h^i, with h t suitable for the particular 
phenomenon (as brilliantly done by McFadden intro the first class of problems [BO] [15], Eigen in the 
middle [BSJ and Burnet in the last class [35] [H]). As for collective influences among agents modeling is 
by far harder. 

In the model we are going to develop, each agent i G (1, ...,V) is endowed with a set of L characters 
denoted by a binary string £j of length L. For example, in social context this string may characterize the 
agent and each entry may have a social meaning (i.e. may take into account an attitude toward 

the opposite sex such that if £f _1 = I, Ui likes the opposite sex, otherwise if £f _1 = 0; in the same way 
£f _2 may accounts for smoking and so on up to L) . In gene networks the overlap among bit strings may 
offer a measure of phylogenetic distance while in immunological context may offer the affinity matrix 
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built up by strings standing for the antibodies (and anti-antibodies) produced by their corresponding 
lymphocytes. 

Now we want to associate a weighted link among two agents by comparing how many similarities they 
share (note that — does not contribute in this scheme, but only 1 — 1), namely 

This description naturally leads to the emergence of a hierarchical partition of the whole population into 
a series of layers, each layer being characterized by the sharing of an increasing number of characters. 
Of course, group membership, apart from defining individual identity, is a primary basis both for social 
and biological interactions and therefore acquaintanceship. As a result, the interaction strength between 
individual i and j increases with increasing similarity. 

Hence, including both terms (one-body and two-bodies) the model we are describing reads off as 

1 V V 

i<j i 

formally identical to the Hopfield model. 

The string characters are randomly distributed according to 

P(tf = +1)= 1 ^, P(tf=0)= 1 ^ a , (2.3) 

in such a way that, by tuning the parameter a G [—1, +1], the concentration of non null-entries for the 
i-th string pi — Ylu^i can ^ e var i e d- When a — > — 1 there is no network and we are left with a non 
interacting spin system, while when a — > +1 we have that = L for any couple and (renormalization 
trough i" 1 apart) we recover the standard Curie- Weiss model. 

Further, when a^O the pattern distribution is biased, somehow similarly to the correlations investigated 
by Amit and coworkers in neural scenarios [13]. Moreover, from Eq.(2.3) we get } = ((l+a)/2)[<5 M „ + 
((1 + a)/2)(l — S^v)], - apart a — which reduces to completely uncorrelated patterns. 

As we will see, small values of a give rise to highly correlated, diluted networks, while, as a gets larger 
the network gets more and more connected and correlation among links vanishes. 

Even though the theory is defined at each finite V and L, as standard in statistical mechanics, we are 
interested in the large V behavior (such that, under central limit theorem permissions, deviations from 
averaged values become negligible and the theory predictive). To this task we find meaningful to let 
even L diverge linearly with the system size (to bridge conceptually to high storage neural networks), 
such that limy_>. 00 L/V = a defines a as another control parameter. Finally, since we are interested in 
the regime of large V and large L we will often confuse V with V — 1 and L with L — 1. 
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3 The emergent network 



The set of strings }i=i,...,v;/i=i,...,i together with the rule in eq. (2.1 1 generates a weighted graph 
Q(V, L,a) describing the mutual interactions among nodes. The following investigation is just aimed at 
the study of its topological features, which, as well known, are intimately connected with the dynamical 
properties of phenomena occurring on the network itself (e.g. diffusion [26, 2, .5], transport [3[7], critical 
properties [311 EL coherent propagation [6], relaxation [44], just to cite a few). We first focus on the 
topology neglecting the role of weights and we say that two nodes i and j are connected whenever Jy is 
strictly positive; disorder on couplings will be addressed in Sec. |3.2| 



It is immediate to see that the number p of non-null (i.e. equal to 1) entries occurring in a string £ is 
Bernoulli-distributed, namely 

»^')=C)(t)'(V) W 

with average and variance, respectively, 

p 0)i =^pPi(p;a,L) = (3.2) 



p=0 



<L=P 2 a,L-pl.L=[ 1 —^-)L. (3.3) 



Moreover, the probability that a string is made up of null entries only is n«=i P(£i = 0) = [(1 — a) /2] L , 
thus, since we are allowing repetitions among strings, the number of isolated nodes is at least V[(l — 
a)/2f. 

Let us consider two strings £j and £j of length L, with pi and Pj non-null entries, respectively. Then, 
the probability P m atch(fc; Pi, Pj, L) that such strings display k matching entries is 

(L\(L-k\(L-p } \ 

, . /T -r \ \k/ \pi — k/ \Oj—k/ /„ 

Pm a tch(k;pi,Pj,L) = — pypy^ — ' ( ) 

which is just the number of arrangements displaying k matchings over the number of all possible ar- 
rangements. As anticipated, for two agents to be connected it is sufficient that their coupling (see eq. 



(2.1)) is larger than zero, i.e. that they share at least one trait. Therefore, we have the following link 



probability 

Pimk(Pi,Pj,L) = y2 P mat ch(k; Pi, Pj , L) = 1 - P ma tch{0;Pi,Pj,L) = 1 Pl (3.5) 

L \ L ~ Pi ~ Pj)- 

The previous expression shows that, in general, the link probability between two nodes does depend on 
the nodes considered through the related parameters pi and p.y. When pi and pj are both large, the nodes 
are likely to be connected and vice versa. Another kind of correlation, intrinsic to the model, emerges 
due to the fact that, given = 1, the node 1 will be connected with all strings with non-null p,-th entry; 



this gives rise to a large (local) clustering coefficient a (see section 3.4). Such a correlation vanishes 
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when a is sufficiently larger than —1, so that any generic couple has a relative large probability to be 
connected; in this case the resulting topology is well approximated by a highly connected, uncorrclatcd 
(Erdos-Renyi) random graph. Moreover, when a — > +1 we recover the fully-connected graph. 



Finally, it is important to stress that, according to our assumptions, repetitions among strings are 
allowed and this, especially for finite L and V, can have dramatic consequences on the topology of the 
structure. In fact, the suppression of repetitions would spread out the distribution Pi(p;a,L), allowing 
the emergence of strings with a large p (with respect to the expected mean value L(l + a)/2); such nodes, 
displaying a large number of connections, would work as hubs. On the other hand, recalling that the 
number of couples displaying perfect overlapping strings is ~ V 2 /2 L , we have that in the thermodynamic 
limit and L growing faster than log V, repetitions among strings have null measure. 



3.1 Degree distribution 



We focus the attention on an arbitrary string £ with p non-null entries and we calculate the average 
probability Pi; n k(p; a) that £ is connected to another generic string, which reads as 



^ Pi(Pi\a, L)P iink (p, pi;L) 

Pi=0 



= 1 - 



I- a 



1 



1 + a 

1 -a 



L-p 



= 1 - 



1 - a 



(3.6) 



This result is actually rather intuitive as it states that, in order to be linked to £, a generic node has 
to display at least a non-null entry corresponding to the p non-null entries of £. Notice that the link 



probability of eq. (3.6| corresponds to a mean-field approach where we treat all the remaining nodes in 



the average; accordingly, the degree distribution P^ cgl . cc (z; p, a,V) for £ gets 



-Pde g rcc(z; P, a, V) 



1 - a 



1 -a 



p(V-z) 



(3.7) 



Therefore, the number of null-entries controls the degree-distribution of the pertaining node: A large p 
gives rise to narrow (i.e. small variance) distributions peaked at large values of z. Notice that Pn n k{p] «) 
and, accordingly, Pd cgrcc (z; p, a,V) are independent of L. 



More precisely, from eq. (3.7 1, the average degree for a string displaying p non-null entries is 



while the pertaining variance is 



z p = V 



V 



1 - 



1 -a 



1-a 



(3.8) 



(3.9) 
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Figure 3.1: Degree distribution -Pdegree(^) for systems displaying small values of L and multimodal 
distribution (upper panel) and large values of L and distribution collapsing into a unimodal one (lower 
panel). In the former case we compare systems of different sizes but fixed a = 0.01, where continuous 



lines represent the analytic estimate of eq. (3.10) while symbols (•) represent data from simulations and 
are reported for clearness only for the case V = 8000. In the lower panel we compare systems with same 
L but different volumes; thicker curves represent Pdegree( z ; o-i L, V), while each mode Pdegree( z ; P, cl, V) 
is depicted in different colors. 



Now, the overall distribution can be written as a combination of binomial distributions 

L 

Pdcgrcc{z;a,L,V) = ^2 P dcgICC (z; p, a,V)P 1 (p; a, L), 

p=0 



(3.10) 



where the overlap among two "modes", say p and p + 1, can be estimated through a p /(z p+ i — z p ) 
Exploiting eqs. (3.8 1 and (3.9) we get 



= 4 1- 



~ 2 



1 -a 



p/2 



1 + a 



(3.11) 



where the generic mode p is confused with p and the approximate result \/L/V was derived by using 
the scaling a = — 1 + j/V 9 , with 1/2 < 6 < 1 (both these points are fully discussed in the next Section); 
also, the last passage holds rigorously in the thermodynamic limit of the high storage regime (L linearly 
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diverging with V). Interestingly, for systems with different scaling regimes among L and V, for instance 
L oc log V [181 [T] , the distribution remains multi-modal because a vanishing overlap occurs among the 
single distributions Pdegree(z; p, a, V): -Pdcgrcc(z; a, L, V) turns out to be an (L + l)-modal distribution 



(see Fig. 3.1 upper panel); vice versa, for L oc V, the overall distribution gets mono-modal (see Fig. 3.1 



lower panel). Briefly, we mention that for 9 = 1/2 the ratio in the l.h.s. of eq. (3.11 1 still converges to a 
hnite value approaching y/a for 7 2 <C a, while for 9 < 1/2 it diverges. 



From eq. (3.101, the average degree for a generic node is 

V L 
Z = ^Z Pdegree^; a, L, V) = Pi(p; a; L)z p = V { 1 

2=0 p=0 



where 



1 + a 



1 - 



1 + a 



(3-12) 



(3.13) 



is the average link probability for two arbitrary strings and which can be obtained by averaging 
over all possible string arrangements, namely, recalling eqs. (3.1) and (3.6), 



X! X! Pi(Pt'i a i L )Pi{Pj'i a i L )P\^{PnPj'i a ' L ) 

Pi=0pj=0 

V ' Pl= Pj =0 



L\ f L — p, t 
Pi 



Pi 



= 1 



1 - 



1 + a 



1 L 



(3.14) 



Of course, eq. (3.14) could be obtained directly by noticing that the probability for the /i-th entries of 
two strings not to yield any contribute is 1 — [(1 + a)/2] 2 , so that two strings are connected if there is 
at least one matching. 



3.2 Coupling distribution 



As explained in Sec. [2j the coupling J{j among nodes i and j is given by the relative number of matching 
entries among the corresponding strings & and Eq. (3.4) provides the probability for £j and £j to 
share a link of magnitude J = k, namely -P C oupiing(^ r ; Pi, Pj, L) = P ma t c h{k; Pi, Pj, L). Following the same 
arguments as in the previous section we get the probability that a link stemming from £j has magnitude 
J, that is 

Pcoupiin g (J; Pi, a) = Pcou P img(J; Pi, pj,L)Pi(pj; a, L) = f P l\ (^-^) (~^r~) ' ( 3 - 15 ) 



Pj=0 



which is just the probability that J out of pi non-null entries are properly matched with the generic 
second node. 
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Similarly to P^ cgicc (z; a, L,V), the overall coupling distribution can be written as the superposition 
P\{p\ a, £)-P CO upiing( J\ Pi a )i giving rise to a multimodal distribution. Each mode has variance a 2 = 
p(l — a 2 )/4 and is peaked at 

J P = P 1 -^, (3.16) 

which represents the average coupling expected for links stemming from a node with p non-null entries. 
Nevertheless, by comparing J p +\ — J p — (1 + a)/2 and the standard deviation yj p(l — a 2 )/2, we find 
that in the limit L = aV and V — > oo the distribution gets mono-modal. 



Anyhow, we can still define the average weighted degree w p expected for a node displaying p non-null 
entries. Given that for the generic node i, w = ■ Jij, we get 



VJ p = Vp 



1 + a 



(3-17) 



~ ,up ■ r 2 

Of course, one expects that the larger the coordination number of a node and the larger its weighted 



degree; such a correlation is linear only in the regime of low connectivity. In fact, by merging eq. (3.8) 

l + a\ log(l- 



and eq. (3.17), one gets 



-V 



(3.18) 



where the last expression holds for z p <^V and a -C 1. 



It is important to stress that (apart pathological cases which will be taken into account in the L — > oo 
scaling later) the variance of p scales as a 2 (a\L) = (1 — a 2 )L/A such that, despite the average of p is 
(1 + a)L/2, substituting p/L with (1 + a)/2 into eq. (3.18) becomes meaningless in the thermodynamic 
limit as the variance of J p diverges as s/L oc \/V : This will affect drastically the thermodynamics 
whenever far from the Curie- Weiss limit. 



It should be remarked that J p represents the average coupling for a link stemming from a node character- 
ized by a string with p non-null entries, where the average includes also non-existing links corresponding 
to zero coupling. On the other hand, the ratio w p /z p directly provides the average magnitude for existing 
couplings. Moreover, the average magnitude for a generic link is 



L 

J = ^Pi(p;a,L)J p = 




(3.19) 



By comparing eq. (3.16) and eq. (3.19) we notice that the local energetic environment seen by a single 
node, i.e. J p , and the overall energetic environment, i.e. J, scale, respectively, linearly and quadratically 
with (1 + a)/2: we will see in the thermodynamic dedicated section that (apart in the Curie- Weiss limit 
where global and local effects merge) despite the self-consistence relation (which is more sensible by 
local condition) will be influenced by V^J, critical behavior will be found at (3 C — J -1 coherently with a 
manifestation of a collective, global effect. 



Anyhow, when V is large and the coupling distribution is narrowly peaked at the mode corresponding to 
Pa,L, the couplings can be rather well approximated by the average value Jl(i+o.)/2 = [(1 + a )/2] 2 = J, 
so that the disorder due to the weight distribution may be lost; as we will show this can occur in the 
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regime of high dilution (8 > 1/2). As for the other source of disorder (i.e. topological inhomogeneity) , 
this can also be lost if a is sufficiently larger than —1 as we are going to show. 



3.3 Scalings in the thermodynamic limit 



In the thermodynamic limit and high-storage regime, L is linearly divergent with V and the average 



probability p for two nodes to be connected (see eq. (3.13)) approaches a discontinuous function assuming 



value 1 when a > — 1, and value when a = — 1. More precisely, as V — > oo there exists a vanishingly 
small range of values for a giving rise to a non-trivial graph; such a range is here recognized by the 
following scaling 

7 

a = - 1 +ye> 

where 9 > and 7 is a finite parameter. 



(3.20) 



First of all, we notice that, following eqs. (3.2) and (3.3), 



«7 

2V^ 



1 ~~ 2V^) ~ P-l+l/V e ,aV, 



(3.21) 
(3.22) 



u -l+y/V e ,aV — 2V 6 - 1 

where the last approximation holds in the thermodynamic limit and it is consistent with the convergence 



of the binomial distribution in eq. (3.1) to a Poissonian distribution. For 9 < 1, p > a, so that when 



referring to a generic mode p, we can take without loss of generality p; the case 9 > 1 will be neglected 
as it corresponds to a disconnected graph. 



Indeed, the probability for two arbitrary nodes to be connected gets 



p=l- 



1 - 



1 + a 



-1 L 



= 1 - 



n aV 



1 - 



4V 2e 



V— >oo 



(3.23) 



so that we can distinguish the following regimes: 



< 1/2, p w 1, z w V => Fully connected (FC) graph 



9 = 1/2, p ~ 1 — e~ 7 Q / 4 ~ 7 2 a/4, z = 0{V) => Linearly diverging connectivity 

Within a mean-field description the Erdos-Renyi (ER) random graph with finite probability G{V,p) 

is recovered. 



1/2<0<1, p-YaV 1 ' 2 " /4, z = 0(V 



Extreme dilution regime (ED) 



In agreement with 



limx 



z = limy^oo z/V = 0. 



9=1, p ~ ~4$r> z — 0(V°) Finite connectivity regime 

Within a mean- field description 7 2 a/4 = 1 corresponds to a percolation threshold. 
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Therefore, while 9 controls the connectivity regime of the network, 7 allows a fine tuning. 



As for the average coupling (see eq. (3.19)) and the average weighted degree: 

J =4^' (3 - 24) 

Now, the average "effective coupling" J, obtained by averaging only on existing links, can be estimated 
as 

r 7 2 /(4V 2e ) if 6* < 1/2 

J = J/p=< 7 2 /[4t/ 2e (l-e^ 2Q / 4 )] if 8 = 1/2 (3.26) 
[ l/(aV) = l/L if 1/2 < ^ < 1 

Interestingly, this results suggests that in the thermodynamic limit, for values of a determined by 



eq. (3.201 with 1/2 < 6 < 1, nodes are pairwise either non-connected or connected due to one sin- 



gle matching among the relevant strings. This can be shown more rigorously by recalling the coupling 



distributions -P C oupiing(</; Pi, L) of eq. (3.15): In particular, for 9 > 1/2, neglecting higher order cor- 
rections, for J = the probability is p ~ exp(a7 2 y 1_2e /4) — 1 - 7 2 a/(4F 2e_1 ), for J = 1/L the 
probability is p\ ~ po^a/^V 29 ^ 1 ) ~ 1 — pq. For 9 = 1/2 this still holds for «7 2 /4 <C 1, which cor- 
responds to a relatively high dilution regime, otherwise some degree of disorder is maintained, being 
that pk ~ («7 2 /4) fc /k\. On the other hand, for 9 < 1/2, while topological disorder is lost (FC), the 
disorder due to the coupling distribution is still present. However, notice that for 9 — and 7 = 2, 
-Pcoupiing^; Pi, L) gets peaked at J = L and, again, disorder on couplings is lost so that a pure Curie- 
Weiss model is recovered. 

This means that, for L = aV and V — > 00, we can distinguish three main regions in the parameter 
space (9, a, 7) where the graph presents only topological disorder (9 > 1/2), or only coupling disorder 
(6> < 1/2), or both (9 = 1/2 A 7 2 a = 0(1)). 

In general, we expect that the the critical temperature scales like the connectivity times the average 
coupling and the system can be looked at as a fully connected with average coupling equal to J or as a 
diluted network with effective coupling J and connectivity given by z; in any case we get ~ J (erf. 
eq.(4.37)). 



3.4 Small-world properties 



Small-world networks are endowed, by definition, with high cluster coefficient, i.e. they display sub- 
networks that are characterized by the presence of connections between almost any two nodes within 
them, and with small diameter, i.e. the mean-shortest path length among two nodes grows logarithmi- 
cally (or even slower) with V. While the latter requirement is a common property of random graphs 
[771 [78] , the clustering coefficient deserves much more attention also due to the basic role it covers in 
biological [HHl |S7J and social networks [SU |53] . 

The clustering coefficient measures the likelihood that two neighbors of a node are linked themselves; a 
higher clustering coefficient indicates a greater "cliquishness" . Two versions of this measure exist [771 178] : 
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global and local; as for the latter the coefficient Cj associated to a node i tells how well connected the 
neighborhood of i is. If the neighborhood is fully connected, q is 1, while a value close to means that 
there are hardly any connections in the neighborhood. 



The clustering coefficient of a node is defined as the ratio between the number of connections in the 
neighborhood of that node and the number of connections if the neighborhood was fully connected. Here 
neighborhood of node i means the nodes that are connected to i but does not include i itself. Therefore 
we have 

(3 ' 27) 

where Ei is the number of actual links present, while Zi(zi — 1) /2 is the number of connections for a fully 
connected group of Zi nodes. Of course, for the Erdos-Renyi graph where each link is independently 
drawn with a probability p, one has c ER = p, regardless of the node considered. 



We now estimate the clustering coefficient for the graph Q(a,L,V), focusing the attention on a range 
of a such that the average number of non-null entries per string is small enough for the link probability 
to be strictly lower than 1 so that the topology is non trivial; to fix ideas and recalling last section 
1/2 < 9 < 1. Let us consider a string displaying p non-null entries, corresponding to the positions 
fix, Hz, fi p , and z nearest-neighbors; the latter can be divided in p groups: strings belonging to the 
j-th group have £ Mj = 1. Neglecting the possibility that a nearest-neighbor can belong to more than 



one group contemporary (in the thermodynamic limit this is consistent with Eq. 3.261, we denote with 



the number of nodes belonging to the j-th group, being £. 



whose average value is z/p 



(which, due to the above assumptions is larger than one). Now, nodes belonging to the same group 
are all connected with each other as they share at least one common trait, i.e. they form a clique; the 
contribute of intra-group links is 



1 9 

-22,rii(ni 



1) 



i=l 




while the contribute of inter-group links can be estimated as 

P / \ 2 



Winter ~ niUjp : 



P, 



(3.28) 



(3.29) 



where p is the probability for two nodes linked to i and belonging to different groups to be connected, and 
the sum runs over all possible (£) couples of groups. Hence, the total number of links among neighbors is 



E = E- n 



{E£=ini£Lini|p+(l- 



= 1 '"' 

1 if i = j and zero otherwise; of course, for p 



■p)Sij] — z}/2, where dij is the Kronecker delta returning 
1 we have E = (z 2 — z)/2 and Cj = 1. 



Now, in the average, the probability p is smaller than p as it represents the probability for two strings 
of length L — 1 and displaying an average number of non-null entries equal to p — 1 to be connected. 
However, for p and L not too small the two probabilities converge so that by summing the two contributes 
in eq. (13. 28b and (13.291) we get 



E 



1 



>P, 



(3.30) 
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Figure 3.2: Upper panel: average link probability p = z/N; Lower panel: difference between the average 
clustering coefficient for Q(V,L,a) and for an analogous ER graph just corresponding to p. Both plots 
are presented as function of a and L and refer to a system of V = 2000 nodes. 



where in the last inequality we used p < z— 1. Therefore, it follows straightforward that Ci is larger than 
the clustering coefficient expected for an ER graph displaying the same connectivity, that is c ER = p. 

From previous arguments it is clear that the SW effect gets more evident, with respect to the ER 
case taken as reference, when the network is highly diluted. This is confirmed by numerical data: 
Fig. |3.2| shows in the lower panel the clustering coefficient expected for the analogous ER graph, namely 
c ER — z/V, while in the upper panel it shows the difference between the average local clustering 
coefficient c = 53i=i C i/V an( i ° ER itself. Of course, when a approaches 1, the graph gets fully connected 
and c c ER 1. 

Finally we mention that when focusing on the low storage regime, a non-trivial distribution for couplings 
can give rise to interesting effects. Indeed, weak ties can be shown [55] to work as bridges connecting 
communities strongly linked up, as typical of real networks [52l|89]. Also, as often found in technological 
and biological networks, the graph under study display a "dissortative mixing" [771 [75], that is to say, 
high-degree vertices prefer to attach to low-degree nodes [55] , 
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4 Thermodynamics 



So far the emergent network has been exhaustively described by a random, correlated graph whose links 
are endowed with weights; we now build up a quantitative thermodynamics on such a structure. 



Once the Hamiltonian i?y((j;£) is given (eq. |2.2[ ), we can introduce the partition function (/?;£) as 
the Boltzmann state lo as 

, e -0Hv(v£) 



and the related free energy as 



"<•» " V(fto. (4 - 2) 



A(fi,a,a)= Jim log Z v (/3; f), (4.3) 



where E averages over the quenched distributions of the affinities £. 

Once the free energy (or equivalently the pressure) is obtained, remembering that (calling S the entropy 
and U the internal energy) 

A(f3, a, a) = — /3/(/3, a, a) = 5(/3, a, a) — (3U(/3, a, a), 

the whole macroscopic properties, thermodynamics, can be derived due the Legendre structure of ther- 
modynamic potentials |67| . 



4.1 Free energy trough extended double stochastic stability 



For the sake of clearness now we expose in complete generality and details the whole plan dealing with a 
generic expectation on £ (i.e. E£ = (1 + a)/2), then, we will study the L — > oo scaling, in which a must 
tend to —1 more carefully. 

With this palimpsest in mind, let us normalize the Hamiltonian (2.2) in a more convenient form for this 
section (i.e. dividing by L the Jy, such that the effective coupling is bounded by 1), and let us neglect 
the external held h which can be implemented later straightforwardly. 

ij M 

As a next step, through the Hubbard-Stratonovick transformation |67[I41|. we map the partition function 
of our Hamiltonian into a bipartite Erdos-Renyi ferromagnetic random graph [5] [47] . whose parties are 
the former built by the V agents and a new one built of by L Gaussian variables z^, fj, € (1, L): 

/+oo L I „ V L 

n^Wexp^V^ES^^J. ( 4 - 5 ) 
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where with FJ^ =1 d^z^) we mean the Gaussian measure on the product space of the Gaussian party. 
Note that, even when L goes to infinity linearly with V (as in the high storage Hopfield model 
due to the normalization encoded into the affinity product of the £'s nor the z-diagonal term contribute 
to the free energy (as happens in the neural network counterpart |23j). neither (but this will be clear at 
the end of the section) there is a true dependence by a in the thermodynamics. 

Furthermore, notice that the graph of the interactions among the two parties is now a simple, and no 
longer weighted, Erdos-Renyi |14j : so we started with a complex topology for a single party and we turned 
this problem in solving the thermodynamics for a simpler topology but paying the price of accounting 
for another party in interaction. The lack of weight on links will have fundamental importance when 
defining the order parameters. 

Another approach to this is noticing that if we dilute -randomly- directly the Hopfield model (i.e. as 
checking for its robustness as already tested by Amit [TU]) we push it on an Erdos-Renyi topology, while 
if we dilute its entries in pattern definitions (due to the Hebbian kernel) we have to deal with correlated 
dilution. 

Consequently (strictly speaking assuming the existence of the V limit) we want to solve for the following 
free energy: 

I r+oo L . I ~B v L 

A(j3,a,a) = ^lim^—Elog^ / JJd/i^) exp \d EE^ ^)- ( 46 ) 

To this task we extend the method of the double stochastic stability recently developed in [23] in 
the context of neural networks. Namely we introduce independent random fields 77^ , i € (1,...,V) and 
Xfj.il- 1 € (1, ...,£), (whose probability distribution is the same as for the £ variables -as in every cavity 
approach-), which account for one-body interactions for the agents of the two parties. So our task is to 
interpolate among the original system and the one left with only these random perturbations: Let us 
use t € [0, 1] for such an interpolation; the trial free energy A(t) is then introduced as follows 

-1 r+00 l 

A{t) = ^v Elo& ^ L n^(v>- ( 4 - ? ) 

r—p- VL L V V L 

■ exp (tJ — 6^0"^ + (1 - t)(*T h c Y ma, + E Q " E x ^ 



where now E = E^E^E X and bi c [with l c € (1, ...,L)], and ci b [with If, G (1, ...,V)] are real numbers 
(possibly functions of /?, a) to be set a posteriori. 

As the theory is no longer Gaussian, we need infinite sets of random fields (mapping the presence of 
multi-overlaps in standard dilution [3] [43] and no longer only the first two momenta of the distributions). 
Of course we recover the proper free energy by evaluating the trial A(t) at t = 1, (A(j3, a, a) = A(t = 1)), 
which we want to obtain by using the fundamental theorem of calculus: 

A(l) = A(Q) + J (dA{t')/dt'^ t _dt. (4.8) 

To this task we need two objects: The trial free energy A(t) evaluated at t = and its £-streaming 
d t A(t). 

Before outlining the calculations, some definitions arc in order here to lighten the notation: taken g as 
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a generic function of the quenched variables we have 



%M = E P{h)g{m b ) = E (T) (^Y h (^)\(%), (4.9) 

= E P(Ic)flC0.) = E (3 ^ (^p)'*ff(tf.), (4-10) 

where P(lb) is the probability that lb (out of V random fields) are active, i.e. r/ = 1, so that the number of 
spins effectively contributing to the function g is lb', analogously, mutatis mutandis, for P(l c )- Moreover, 
in the last equation we summed over the probability P(l) that in the bipartite graph a number I of 
links out of the possible V x L display a non-null coupling, i.e. £ ^ 0; interestingly, eq. (4.10) can be 
rewritten in terms of the above mentioned P(lb) and P{l c ). In fact, can be looked at as an V x L 
matrix generated by the product of two given vectors like rj and x, namely = rjiXfj,, in such a way 
that the number of non-null entries in the overall matrix £ is just given by the number of non-null entries 
displayed by r\ times the number of non-null entries displayed by X- Hence, P(l) is the product of P(h) 
and P(l c ) conditional to lbl c = I. 



4.2 The 'topologically microcanonical" order parameters 



Starting with the streaming of eq. (4.7 1, this operation gives raise to the sum of three terms A + B + C. 
The former when deriving the first contribution into the exponential, the last two terms when deriving 
the two contributions by all the 77 and x- 



A 



B 



V.L 



V V LV 



E|X>m^) = -I> 



1 + a 



L = l 



2 

1 + a 



V.L 



V 



E p (k)M h 



h=0 



c = -E^E e x^) = -v^Eq. 

l b = l p h = l 

where we introduced the following order parameters 

I v 



1 + a 



E 



l a =0 



1 L 

Nic = ^E W ^+ 1 ^' 



(4.12) 
(4-13) 
(4-14) 

(4.15) 
(4.16) 



and the Boltzmann states u>k are defined by taking into account only k terms among the elements of the 
party involved. 
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Of course the Boltzmann states are no longer the ones introduced into the definition (4.2) but the 
extended ones taking into account the interpolating structure of the cavity fields (which however will 
recover the originals of statistical mechanics when evaluated at t = 1). 

Namely, u>i b +i has only 4 + 1 terms of the type ba in the Maxwell-Boltzmann exponential, ultimately 
accounting for the (all equivalent in distribution) 4 + 1 values of r\ = 1, all the others being zero. 
In the same way w; c +i has only l c + 1 terms of the type cz in the Maxwell-Boltzmann exponential, 
ultimately accounting for the (all equivalent in distribution) l c + 1 values of x = 1, all the others being 
zero. 

When dealing with ^ we can decompose the latter accordingly to what discussed before. By these 
"partial Boltzmann states" we can define the averages of the order parameters as 

v-i 



(M) = ^Pft)^, (4.17) 

h 

L-l 

(N) = ^P(/ c )iV ;c . (4.18) 



These objects may deserve more explanations because, as a main difference with classical approaches 
[5] |39) [4"5] . here replicas and their overlaps are not involved (somehow suggesting the implicit correct- 
ness of a replica symmetric scenario). Conversely, we do conceptually two (standard) operations when 
introducing our order parameters: at first we average over the (i-extended) Boltzmann measure, then 
we average over the quenched distributions. Let us consider only one party for simplicity: during the 
first operation we do not take the whole party size but only a subsystem, say k spins (whose distribution 
is symmetric with respect to for both the parties, — 1, +1 for the dichotomic, Gaussians for the con- 
tinuous one). Then, in the second average, for any k from 1 to the volume of the party, we consider all 
the possible links among these k nodes in this subgraph. As the links connecting the nodes are always 
constant (i.e. equal to one due to the Hubbard-Stratonovich transformation (4.4)) in the intensity, the 
resulting associated energies are, in distribution and in the thermodynamic limit, all equivalent: We are 
introducing a family of microcanonical observables which sum up to a canonical one, in some sense close 
to the decomposition introduced in |22) . 



4.3 The sum rule 



Let us now move on and consider the following source S of the fluctuations of the order parameters, 
where M; b , 7V ic stand for the replica symmetric values^] of the previously introduced order parameters: 

S = (^-) ^EEwf^-^J^-JV,,)) (4.19) 



2 

1 + a 



aP{[M - M)[N - N)). (4.20) 



4 strictly speaking there are no replicas here but configurations over different graphs. However the expression RS- 
approximation, meaning that we assume the probability distribution of the order parameters delta-like over their average 
(denoted with a bar) is a sort of self-averaging and is an hinge in disordered statistical mechanics such that we allow 
ourselves to retain the same expression with a little abuse of language. 
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We see that with the choice of the parameters fe; c = y/a/3Ni c and ci b = J (3/aM[ b , we can write the 
i-streaming as 

1 4- _V-1L-1 

i = S-^^^ P(l b )P(l e )M lb N lc . 

The replica symmetric solution (which is claimed to be the correct expression in diluted ferromagnets) 
is simply achieved by setting S — and forgetting it from future calculations. 

We must now evaluate A(0). This term is given by two separate contributions, each for each party. 
Namely we have 

+oo L V L 

^2l b = l C lb ^m-^ 



A(0) = — E log E e E ^ =1 bl " Ei + — E log / j [ d^z^h 

' ' J -co . n 



ju=0 

log 2+ (l±^) j2p(i b )J2p(i c ) log cosh (v^# Ie ) + (~) 2 f E p (W' 



i 6 =o z c =o 

Summing A(0) plus the integral of dt[A(S = 0)] we finally get 

L-l v-i 



(1 4- \ — 
— - £ E P (W^) log cosh (y^PNu) (4.21) 

/3 A + ^'gp^j^-i^^^gp^gM,^. (4.22) 

^ z c 

It is possible to show that (as each bipartite ferromagnetic model [23] |48j ) the free energy obeys a 
min-max principle by which, extremizing the free energy with respect to the order parameters we can 
express (N) trough (M): The trial replica symmetric solution, expressed trough (M),(N) is (at fixed 
(N)) convex in (M). This defines uniquely a value (M(N)) where we get the max. Further, (M(N)) is 
increasing and convex in (-/V 2 ) such that the following extremization is a well defined procedure. 

k k l c k 

dA 



Due to the mean field nature of the model, as we can express Nk trough the average of the Mk, we can 
write the free energy of our network trough the series of M[ b alone [as expected as we started by eq. 

AGS, a) = log2+(^) log cosh (tanh- 1 [E P (W^j) ( 4 - 25 ) 

h 

+ { (^) 2 E p (^ - (^)E p (w> tanh_1 (E p (w ; 

lb h l' b 

As anticipated there is no true dependence by a. Note that without normalizing the scalar product 
among the bit strings we should rescale accordingly with a, as in the L — > oo limit we would get an 
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infinite coupling (which is physically meaningless). 

Before exploring further properties of these networks, we should recover the well known limit of Curie- 
Weiss a = +1 and isolate spin system a = — 1. 

Let us work out for the sake of clearness the self-consistency in a purely Curie- Weiss style by extremizing, 
with respect to (M, ) eq. (4.25): 



d { M)A{P) = ( : 



(M) 



(M) = tanh(/3( 



1 



){M)) = 



){M) -(^) 



tanh 



1 



(M) 



1 - {MY 



tanh _1 (M) 







)(M>, 



(4.26) 



2 - 1 (M)=^ 

such that the to get the classical magnetization in our model we have to sum overall the contributing 
graphs, namely (Mew) = (M) = Y^i P{h)Mi b , an d we immediately recover 

a-t-1 => A(/3,a = -l) = log2, (4.27) 



-1 A(/3, a = +1) = log 2 + logcosh(/3(M)) — ^(M 



(4.28) 



which are the correct limits (note that in eq. (4.27) J — 0, while in eq. ( 4.28| ) J = 1). 

Furthermore, we stress that in our "topological microcanonical" decomposition of our order parameters, 

when summing over all the possible subgraphs to obtain the CW magnetization, these are all null 

apart the only surviving of the fully connected network, so the distribution of the order parameters 

becomes trivially cc 5(M — Mcw\ namely, only one order parameter survives, the classical Curie- Weiss 

magnetization. 



4.4 Critical line trough fluctuation theory 



{(OMAf) - (0)(MN)) - N{(OM) - (0)(M)) - M((ON) - (0)(N)) , (4.29) 



Developing a fluctuation theory of the order parameters allows to determine where critical behavior 
arises and, ultimately, the existence of a phase transition^] 

To this task we have at first to work out the general streaming equation with respect to the t-flux. 
Given a generic observable O defined on the space of the cr, z variables, it is immediate to check that the 
following relation holds (we set a = 1 for the sake of simplicity as it never appears in the calculations 
(as can be easily checked by substituting (N) with (M) trough eq. (4.23) which changes the prefactor 
from (^^)^faj3 — > (^^) 2 /? and express the fluctuations only via the real variables trFj): 

d(Q) _ 1 + a 
dt ~ ~Y 

where we defined the centered and rescaled order parameters: 

(M) = WJ2 P ( l >>)( M h-Mi b ) = VV(M-M), (4.30) 
h 

(AO = Vl^2p{I c )(N-N) = VZ(N-N). (4.31) 

la 

5 Strictly speaking this approach holds only for second order phase transition, which indeed is the one expected in 
imitative models, even in presence of dilution [3]. 

6 Another simple argument to understand the useless of a is a comparison among neural networks: in that context, a 
rules -in the thermodynamic limit- the velocity by which we add stored memories into the network with respect to the 
velocity by which we add neurons. If the former are faster than a critical value, by a TLC argument they sum up to a 
Gaussian before the infinite volume limit has been achieved and the Hopfield model turns into an SK 23 . Here there is no 
danger in this as we have only positive -normalized- interactions. 
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Now we focus on their squares: We want to obtain the behavior of (A4 2 ) t =i, (MJV) t =i, (A/" 2 ) t= i, so to 
see where their divergencies (onsetting the phase transition) are located. 
By defining the dot operator as 

(6) = ( 1 -^)^d t (0) (4.32) 



N(M 2 )(M) - M{M 2 N) + M{M 2 Af) 



we can write 




{M 2 ) = 


(M 3 AT) - 


(MAT) = 


(M 2 Af 2 ) - 


(AT 2 ) = 


(Af 3 M) - 



Now, for the sake of simplicity, let us introduce alternative labels for the fundamental observables. We 
define A(t) = (M 2 ) t , D(t) = (MN) t and G(t) = (W 2 ) t and let us work out their t — value, which 
is straightforward as at t = everything is factorized (alternatively these can be seen as high noise 
expectations) : 



A(t = 0) = 1, D(t = 0) = 0, G(t = 0) 



. 1 + a. o , - 9, 
1 + " (M 2 

2 a 



(N 2 



1 



where we used the self-consistence relation (4.23) and assumed that at least where everything is com- 
pletely factorized the replica solution is the true solution^] Following the technique introduced in |51j . 
starting from the high temperature and, under the Gaussian ansatz for critical fluctuations, we want to 
take into account correlations among the order parameters. Within this approach, using Wick theorem 
to split the four observable averages in series of couples, the (formal) dynamical system reduces to 



A(t) = 2A(t)D{t), 

D(t) = A(t)G(t) + D 2 (t), 

G(t) = 2G(t)D(t). 



(4.33) 
(4.34) 
(4.35) 



We must now solve for A(t),D(t),G(t) and evaluate these expression at t = ^t^V? accordingly to the 



definition of the dot operator in eq.(4.32|. Notice at first that 

A 



d t log A 



A 



2D 



G 
G 



dtlogG. 



This means dt(A/G) = and as A(0)/G(0) = 1 we already know that A(t) = G(t): the fluctuations of 
the two order parameters behave in the same way, not surprisingly, as already pointed out their mutual 
interdependence several times. 
We are left with 



D(t) = G 2 {t) + D 2 (t), 
G(t) = 2G(t)D(t). 



(4.36) 
(4.37) 



By defining Y = D + G we immediately get, summing the two equations above: Y = Y 2 by which we 
get Y(t) = Y(0)/(l - tY(0)). As Y(0) = 1 we obtain that 



D(t=( 



1 



-)^) + G(t = (- 



WP) = 



I 



i-(W 



7 Any debate concerning RSB on diluted ferromagnets is however ruled out here as we are approaching the critical line 
from above. 
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so there is a regular behavior up to /3 C — l/(i±^) 2 . 

We must now solve separately for D and G: this is straightforward by introducing the function Z = G _1 
and checking that Z obeys 

-Z -2YZ + 2 = 0, 

which, once solved with standard techniques (as Y is known) gives G(t) — [2(1 — i)]" 1 and ultimately, 
simply by noticing the divergencies of A(t = (1 + a)^/2),D(t = (1 + a)V/J/2),G(t = (1 + a)y/]3/2), 
we get the critical line for both the squared order parameters and their relative correlation: All these 
functions do diverge on the line 

defining a phase transition according with intuition. 



4.5 L — y oo scaling in the thermodynamic limit 



As we understood in Section (3.3), in the V —y oo and L —y oo limits we need to tune the limit of 
a — y — 1 carefully to recover the various interesting topologies and to avoid the trivial limits of fully 
connected/disconnected graph. 

In particular, a must approach —1 as a = —1 + j/V e . To tackle this scaling it is convenient to use 
directly 7, 9 as tunable parameter and rewrite the Hamiltonian in the following formfj^] 

1 V L \fWfa 

Hv (°"> = 2aV 2 ( 1 - f} ) ^ ^ $tj a i a i Hv,l{°i z ; = -yl=r zZ ZwCiZi*' ( 4 - 39 ) 

ij n i,fi 

where the difference among the two expression H, H is due to the Hubbard-Stratonovick transformation 
applied to the coupled partition functions, as performed early trough eq.(4.4). 
Our free energy reads off now as 

A(a, 0,7,0) = v lim o iElog^ j JJd^(z p ) exp(^^) X>/^> ( 4 - 4 0) 

where a accounts for the different ratio among the two parties, /3 the noise into the network, 9 selects 
the graph (see Sec. (3.3)) and 7 is the fine tuning inside the chosen topology. 

The interpolating scheme remains the same: we introduce the right amount of random fields and use 
t € (0, 1) to define 

A(t) = —ElogY, / d^)exp[tH v , L (a,z^) + (l-t)(Y j ari i a i +Y,h^)}- (4-41) 

(7 i /J, 

By performing the t-streaming easily we get 

8 t A(t) = — ^-((M - M)(N - AO) + y^MN, (4.42) 



8 As we are going to see soon it is not possible to normalize the Hamiltonian -both the coupling strength and the volume 
extensiveness- for all the possible graphs in only one expression. We choose to normalize so to tackle immediately the 
better known limits, however apparent divergencies in the couplings develop and can be standardly avoided by properly 
rcscaling the temperature corresponding to the amount of nearest neighbors, as in more classical approaches. 
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such that the replica symmetric sum rule gets 



A(l) = A(0) - ^p-MN, (4.43) 
and the replica symmetric free energy reads off as 

AGS, 7,0) = log 2+ ^logcosh^iVy ) + ^M 2 - ^MN. (AAA) 

Let us now investigate some limits of this expression and its self-consistency. Note that by extremizing 
with respect to the order parameters we can skip N trough M as 

(N) = ^(M). (4.45) 



4.5.1 = case: Fully connected, weighted and Curie- Weiss scenario 

The case = reduces to a fully connected graph, and in particular in the upper bound for 7 (i.e. 
7 = 2) its topology recovers the unwcighed CW model (see sec. 3.3). We should recover here even the 
CW thermodynamics. 

A(/3, 7 ,0 = 0) = log 2 + |logcosh(^|(M)) - ^(M> 2 , (4.46) 
and its self-consistency relation reduces to 

(M)=tanh(^|(M)). 

This holds generally for the weighted graph; in particular when 7 = 2 =>■ J = 1 and the graph gets 
un- weighted (still fully connected), we get the standard Curie- Weiss limit once more: 

A(/3,7 = 2,0 = O) = log2 + logcosh(/3(M)) - ^2(M) 2 , (4.47) 
(M) = tanh(/3(M)). (4.48) 



4.5.2 = 1/2: Standard dilution and Erdos-Renyi scenario 

With a scheme perfectly coherent with the previous one we can write down free energy and its coupled 
self-consistency as 

AGS, 7, = 1/2) = y li^(log2+ 2 ^=logcosh(^^(M))-^ 2 (M) 2 ), (4.49) 
(M) = lim tanh(^Vv(M)). (4.50) 

Let us stress that, as \fj = — the argument of the logarithm of the hyperbolic cosine scales as 
VJV(M): This is coherent with the lack of a proper normalization into the Hamiltonian (4.39) because 
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for 6 = 1/2 the latter is still divided by V which should not appear. To avoid the lack of a universal 
normalization, we need to renormalize the local average coupling by a factor V so that we get the correct 
behavior, namely we write explicitly the free energy putting in evidence that p ~ 1 — exp(— aj 2 

A0,j,6 = 1/2) = log 2 + \/jlogcosh(^(Af)) - ^(M) 2 , (4.51) 

P 2p 

such that, being j3 = j3pV [3], we can easily recover the trivial limits of the CW case when p — > 1 (and 
choerently J — > 1, /? — > f3) and of the fully disconnected network p — >• A($, 7 — > 0, 9 = 1/2) = log 2 
as p is superlinear in 7. 



4.5.3 = 1: Extreme diluted regime 



With a scheme perfectly coherent with the previous one we can write down free energy and its coupled 
self-consistency as 



AG8 )7 ,0 = 1) = ^(^2+^ log cosh(^(M))-^(M) 2 ), (4.52) 

(M) = lim tanh(^-F(M)). (4.53) 

v— ¥00 2 

Of course here, with respect to the previous case, we get even stronger divergencies. Now we need to 
renormalize the local average coupling by a factor V 2 . 



4.6 Numerics: Probability distribution 



As the critical line is obtained, in the fluctuation theory, through the Gaussian ansatz, we double check 
our finding via numerical simulations. 



First of all, we notice that since the interaction matrix J^- is symmetric (Jjj = Jji), detailed balance 
holds and it is well known [69 , 4 how to introduce a Markov process for the dynamical evolution ruled 
by Hamiltonian (2.2) and obtain the transition rates for stationarity: Montecarlo sampling is then 
meaningful for equilibrium investigation. 



The order parameter distribution function has been proved to be a powerful tool for studying the critical 
line in different kinds of systems; in particular, for magnetic systems the order parameter can be chosen 
as the magnetization per spin which, in finite-size systems, is a fluctuating quantity characterized by 
a probability distribution P(m) [37]. In Ising-like models undergoing a second-order phase transition 
it is known that at temperatures lower than the critical temperature Z?^ 1 , the distribution P(m) has a 
double peak, centered at the spontaneous magnetization +m and —m. At temperatures greater than 
/3" 1 , P(m) has a single peak at zero magnetization, and exactly at /3" 1 a double peak shape is observed. 

In Fig. |4.1| we plotted numerical data for the probability distribution, obtained by means of Monte Carlo 
simulations, where P(m) corresponds to the fraction of the total number of realizations in which the 
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system magnetization is m. In the main figure we show the distribution for a system with V = 1000 set at 
a temperature /3 _1 = 1.1/3" 1 , while in the inset we compare system of different sizes set at a temperature 
= 2/3 f T 1 . Notice that, for such small temperatures, as the size in increased the distribution is more 
and more peaked, while the probability to have zero magnetization is vanishing; this corroborates the 
replica-symmetric ansatz. 



5 Conclusion 



In this paper we pioneered an alternative way for obtaining complex topologies. Interestingly from our 
approach small world features are emergent properties and no longer imposed a-priori, furthermore the 
core-theory descents from a simple shift — 1 — > in the definition domain of the patterns of an Hopfield 
model and is able to recover all the best known complex topologies. 

From a graph theory perspective we introduced a model which, given a set of V nodes, each corresponding 
to a set of L attributes encoded by a binary string £, defines an interaction coupling Jij = • £j) for any 
couple of nodes The resulting system can be envisaged by means of a weighted graph displaying 

non trivial correlations among links. In particular, when attributes are extracted according to a discrete 
uniform distribution, i.e. P(^) = (l+a)/2 for any i G [1, V] and /z £ [1, L], being a a tunable parameter, 
we get that when a is sufficiently small the resulting network exhibits a small-world nature, namely a 
large clustering coefficient; As a is spanned, the network behaves as an isolate spin system, an extreme 
dilute network, a linearly diverging connectivity network a weighted fully connected and an un-weighted 
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fully connected network, respectively. Moreover, nodes are topologically distinguishable according to 
the concentration p of non-null entries present in their corresponding binary strings: interestingly, if the 
scaling among L and V is sub-linear (i.e. P oc \nV or even slower as in low storage networks [HUB]) the 
degree distribution turns out to be multi-modal, each mode pertaining to a different value of p. Instead, 
whenever the scaling is (at least) linear -L oc V-, the distribution gets mono-modal. At least numeri- 
cally, at finite V, L, when looking at the distribution of weights, one finds that weak-ties work as bridge, 
in full agreement with Granovetter theory: indeed, one can detect small highly-connected clusters or 
" communities" made up of nodes with similar attributes and links connecting different communities are 
found to correspond a small coupling. 

Then, as diluted models are of primary interest in disordered statistical mechanics, by assuming self- 
averaging of the order parameters, we solved the thermodynamic of the model: this required a new 
technique (a generalization to infinitely random fields of the double stochastic stability) which is of 
complete generality as well and paves another way for approaching dilution in complex systems. 
Furthermore, within this framework, replicas are not necessary and instead of averaging over these copies 
of the system (and dealing with the corresponding overlap) we can obtain observables as magnetization 
averages over local subgraphs, implicitly accounting for a replica symmetric behavior (which is indeed 
assumed trough the study). 

An interesting finding, on which both the investigations converge (graph theory and statistical mechan- 
ics), is a peculiar non-mean field effect in the overall fields felt by the spins: from eq. (3.17) we see 
that the field insisting on a spin scales as \fj while the averaged field on the network scales as J -see 
eq. (3.20)- (which is the canonical mean field expectation). Furthermore, looking at eq. (4.25) we 
see that in the hyperbolic tangent encoding the response of the spin to the fields, the contribution of 
the other spins is not weighted by J but by \f~J . As in the thermodynamics the coupling strength has 
been normalized, J < 1 — > VJ > J: in complex thermodynamics there is a super-linearity among the 
interactions: despite this does not affect the critical behavior which is a global feature of the system 
and consequently is found to scale with J (see eq. (4.37)), this may substantially change all the other 
speculations based on intuition. 

Of course, in the Curie- Weiss limit this effect disappear as global and local environments do coincide 
(i.e. J= 1). 

It is worth stressing that (microscopic) correlation among bit-strings is directly related to macroscopic 
behavior (e.g. critical line), providing a new intriguing mechanics to study the former via investigations 
on the latter (e.g. in social networks, gene regulatory networks, or immune networks). 
Next step in the research now should be double directed: from one side, a clear statistical mechanics 
of scale free networks may stem from our approach. From the other side, applications of this theory to 
real systems (first at all a clear investigation on dynamical retrieval properties), both in biology and in 
sociology, should be a primary challenge as well. 
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